The present invention relates to machine learning. In particular, the present invention relates to reducing computation times associated with probabilistic functions used in machine learning.
When performing automated recognition tasks, such as speech recognition or image recognition, it is common to compare an input signal to a probabilistic model to identify a most likely phone or image given the input. One factor that can complicate this comparison is that the data in the input signal may represent a phone or an image that has gone through some type of transformation such as a translation in which the data is shifted either temporally or spatially. For example, for speech signals, the pitch of the phone may be different than the pitch of the phones used in the models. For image signals, the image of interest may be shifted relative to the images used to train the models.
To overcome this problem, the art has developed transformation invariant models that treat the transformation as a hidden variable during construction of the models. Under such systems, a model must be built for both the transformations and the phone or image at the same time. This is typically done using an Expectation-Maximization algorithm in which the parameters of the transformation model are estimated during the Expectation step and then are used to estimate the phone or image model during the Maximization step. The phone or image model is then used to re-estimate the transformation model. This iteration continues until the two models reach some stable point.
While this technique can be used to build small transformation invariant models, it cannot be used for larger models because the calculations become intractable. For example, to estimate an image model for a display having 10,000 pixels, 108 scalar calculations must be performed for each training image that will be used to create the image model.
Thus, a technique is needed for training transformation invariant models without requiring as many calculations as found in the current state of the art.